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Traditionally, frequency dependent evolutionary dynamics is described by deterministic replicator dynamics 
assuming implicitly infinite population sizes. Only recently have stochastic processes been introduced to study 
evolutionary dynamics in finite populations. However, the relationship between deterministic and stochastic 
approaches remained unclear. Here we solve this problem by explicitly considering large populations. In par- 
ticular, we identify different microscopic stochastic processes that lead to the standard or the adjusted replicator 
dynamics. Moreover, differences on the individual level can lead to qualitatively different dynamics in asym- 
metric conflicts and, depending on the population size, can even invert the direction of the evolutionary process. 
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Darwinian evolution represents an intrinsically frequency 
dependent process. The fitness or reproductive output of an 
individual is not only linked to environmental conditions but 
also tightly coupled to the type and frequency of its com- 
petitors. Evolutionary game theory has become 
a powerful framework to investigate the evolutionary fate of 
individual traits with differing competing abilities. Consider 
a population of two types A and B. The fitness (or payoff) 
of the two types depends on their interaction partners and is 
determined by the payoff matrix 
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Traditionally, the dynamics of such systems is investigated in 
the context of the well-known replicator equation |^ Q], 
a deterministic differential equation describing the change in 
frequency of the two (or more) types in infinite populations. 
For two types, this results in four basic scenarios of coevo- 
lutionary dynamics jj^, while complex dynamics can arise in 
higher dimensions la] . 

In nature, however, populations are finite in size and the 
deterministic selection process is augmented and disturbed by 
stochastic effects and random drift. This has long been recog- 
nized by population geneticists and goes back to the seminal 
work by Wright |7] and Fisher fs']. Assuming a finite but con- 
stant population size the balance between selection and drift 
can be described by the Moran process 0. The microscopic 
dynamics consists of three simple steps: (i) selection, an indi- 
vidual is randomly selected for reproduction with a probabil- 
ity proportional to its fitness; (ii) reproduction, the selected in- 
dividual produces one (identical) offspring; (iii) replacement, 
the offspring replaces a randomly selected individual in the 
population. 

The Moran process allows to derive the fixation probability 
of mutant genes or investigate the effect of population struc- 
tures on the fixation probability fioll . Originally, the Moran 
process was formulated in a frequency independent setting 
where the fitness of an individual is genetically determined 
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and remains unaffected by interactions with other individuals 
as in Ref. fTHl . Only recently, the frequency dependent ap- 
proach of evolutionary game theory and the Moran process 
have been successfully combined in order to investigate the 
evolutionary dynamics in finite populations OEl. The fit- 
ness of an individual now comprises two components: the fre- 
quency independent baseline fitness which is associated with 
genetic predisposition and the frequency dependent contribu- 
tion from interactions with other members of the population. 

Thus, evolutionary dynamics can be described by a continu- 
ous deterministic replicator equation or by a stochastic micro- 
scopic description of a birth-death process such as the Moran 
process. So far, the relation and transition between the two ap- 
proaches remained unclear. We show that different replicator 
dynamics are associated with different microscopic processes 
and moreover that the dynamics derived for infinite popula- 
tions may undergo qualitative changes in finite populations. 

If every individual interacts with a representative sample 
of the population, the average payoff of A and B individuals 
will be determined by the fraction of coplayers of both types. 
Excluding self interactions, this leads to the payoffs 
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where i is the number of A individuals and is the population 
size. 

The effective reproductive fitness of an individual of type 
k is then given by 1 — w + wtt^ where w determines the 
relative contributions of the baseline fitness, which is conve- 
niently normalized to one, and the payoffs result from interac- 
tions with other individuals 1 12]. Thus, w coiTesponds to the 
strength of selection acting on the game under consideration. 
Note that any w <1 can be mapped to a system with a differ- 
ent payoff matrix and w = 1 ] 14]. For the Moran process, the 
probability that the number of A individuals increases from i 
to i + 1 is 

r+(,) = l.z^!i±^4^, (3) 



1 



• + w{■K^)N N 



2 



whereas it decreases from i to z — 1 with probability 
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{jTi) = (^nf i + nf {N — i)) /N is the average payoff in the 
population. 

Note that the selection mechanism in the Moran process re- 
quires perfect global information on the current state of the 
population. This is a very strong requirement and in many 
situations undesirable. Therefore, we propose an alternative 
formulation for the microscopic processes entirely based on 
local information: In each time step, a randomly chosen indi- 
vidual b compares its payoff to the payoff of another randomly 
chosen individual a. It switches to the other's strategy with 
probability 
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where ATTmax is the maximum possible payoff difference and 
< w < 1 measures the strength of selection. Note that in 
contrast to the Moran process, the local update is invariant un- 
der linear rescaling of the payoff matrix. The transition matrix 
for the number of A individuals in this process i is then given 
by 
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In both process, the number of A individuals remains constant 
with probability T"{i) = 1 - T+(i) - T-{i). Further, the 
states i ~ and i = are absorbing. 

We can directly compute the probability that a single A in- 
dividual fixates in the population, <pA- In general, this proba- 
biUty is given by JTsIl 
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In the limit of weak selection, i/; <C 1, the fixation probabili- 
ties can be computed analytically The fixation probabil- 
ity for the Moran process is higher than for the local update 
mechanism if AvTmax > 2. 

The stochastic process can be formulated in terms of the 
master equation 1 17, 18| 

P^+\i) - p-'ii) = P^i-l)T+{i-l)-P^{i)T-{i) 
+P^{i + l)T-{i + 1) - P^{{)T+{i) 

where P'^{i) is the probability that the system is in state i at 
time r. Introducing the notation x = i/N, t ^ t/N and the 
probability density p{x, t) = N P'^{i) yields 

p{x,t + N-^) - p{x,t) = p{x - N-\t)T+{x - N-^) 

+p {x + N-\t)T-{x + N-^) 
-p{x,t)T-{x) - p{x,t) T+{x). 



For A'^ ^ 1, the probability densities and the transition proba- 
bilities are expanded in a Taylor series at x and t. Neglecting 
higher order terms in A^^^ we get 1 17] 
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[a{x)p{x,t)] + -^ [b\x)p{x,t)] , (8) 



with a{x)^T+{x)-T-{x) and b{x) = ^ j^[T+{x) + T-{x)]. 
Note that, for large but finite A^, this equation has the form of a 
Fokker-Planck equation. Since the internal noise is not corre- 
lated in time as subsequent update steps are independent, the 
Ito calculus 1 17| can be applied to derive a Langevin equation 
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where f is uncorrected Gaussian noise and b{x) ~ for 
X = and x — 1. The multiplicative noise term leaves the 
absorbing nature of the boundaries unaffected. Qualitatively 
similar results are obtained by introducing noise in the payoff 
matrix In contrast, to account for stochastic effects 

through additive noise |21, 22] is problematic because of the 
boundaries of x. 

For N ^ oo, the diffusion term b(x) vanishes with — 1= and 
a deterministic equation is obtained. For the Moran process, 
this yields 
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where tt^{x) = xa + {1 — x)b, n^{x) ^ xc + {1 — x)d, and 
{tt{x)) = xt:^{x) + (1 - x)tt^{x). T = ^ is essentially 
the baseline fitness. Equation ilO\ is the adjusted replicator 
dynamics introduced in Ref. ]3]; an alternative derivation for 
imitation dynamics is given in (2.31 . 

For the local update mechanism, we find 
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N^oo \ ATTmax A^ A^ 

— Kx (^tt'^{x) — {n{x))) , 
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where k = -r-^^^ — is a constant factor influencing the 
timescale only. Equation ( II II is the standard replicator equa- 
tion and represents the traditional approach to evolutionary 
dynamics in infinite populations ]4]. The difference between 
the two dynamics amounts to a dynamic rescaling of time 
which leaves the fixed points unchanged. Nevertheless, the 
differences in the microscopic updating can give rise to sub- 
stantial differences in the macroscopic dynamics. To illus- 
trate this, let us consider two famous examples: the Prisoner's 
Dilemma fl^ and Dawkins Battle of the Sexes fl^. 

The Prisoner's Dilemma describes the problem of coopera- 
tion where two players simultaneously decide whether to co- 
operate (C) or defect (D). Cooperation incurs costs c but pro- 
duces a benefit b to the other player (b > c) whereas defection 
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neither costs nor benefits anyone. In this game defection is 
dominant because defectors are better off no matter what the 
other player does. Thus, rational players end up with nothing 
rather than earning the more favorable payoff for mutual co- 
operation — hence the dilemma. The Prisoner's Dilemma is 
determined by the payoff matrix 
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is given by x 
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x{l — x), where x is the fraction of 



defectors. The solution yields 



x{t) = XQ \xo + (1 - :Eo)e-"'*^/(''+^)' 
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with a;(0) = xq and x{t oo) — x* — 1. The adjusted repli- 
cator dynamics can be solved only numerically for the given 
payoff matrix and general w. For w > 0, Fig.^illustrates that 
the convergence to a;* 1 is faster for the adjusted replicator 
dynamics as compared to the standard replicator dynamics. 

Dawkins Battle of the Sexes f23\2^ is a cyclic game refer- 
ring to asymmetric conflicts in parental care: If males (cT) are 
philanderers (B^), it pays for females (5) to be coy (A^), in- 
sisting on a long courtship period to make males invest more 
in the offspring. However, once most males are faithful (A^^), 
fast females are favored (B^) avoiding the costs of courtship. 
Subsequently, the male investment into the offspring is no 
longer justified, philanderers are again favored (B^^), and the 
cycle continues. This is characterized by the payoff matrix 
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where the first element is the payoff of the males and the sec- 
ond element is the payoff of the females 1311 . Note that this 
game is also called 'Matching Pennies' 122112811 . 

The dynamics of Dawkins Battle of the Sexes is qualita- 
tively different for the adjusted and the standard replicator dy- 
namics 1 3]. Comparisons of the deterministic dynamics to the 
corresponding stochastic process in finite populations reveal 
further interesting differences. For the standard replicator dy- 
namics H — —x{l — x)y{l — y), where x is the fraction of 
faithful males and y the fraction of coy females, is a constant 
of motion which measures the distance from the Nash equilib- 
rium q = (|, ■^). Consequently, q is a neutrally stable fixed 
point surrounded by periodic orbits. Figure|2]shows that this is 
a spurious result valid only in the hmit TV — Ng — 00. 
For any finite N, H > holds and every trajectory spirals 
away from q. Note that in this case H is approximated by 
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where we added c in order to avoid negative payoffs. Figure 

^compares the evolution from a state with a small fraction of § 

defectors into the evolutionary end state with defectors only. 2 10- 

The standard replicator dynamics for the Prisoner's Dilemma q 
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FIG. 1 : Prisoners Dilemma: (a) Approach to the Nash equilibrium 
from an initial state with 10% defectors for simulations of the Moran 
process (o) and the local update rule (A) with A'^ — 200 as com- 
pared to numerical solutions of the corresponding adjusted replicator 
dynamics (dashed line) and the standard replicator dynamics (solid 
line), respectively. The dynamics based on the Moran process con- 
verges much faster to the equilibrium state x* = 1. (b), (c) Con- 
vergence of the stochastic simulations (o, A) and the corresponding 
Langevin equations (dashed and solid lines) to the deterministic ad- 
justed and standard replicator dynamics for increasing population 
sizes A'^. (b) Deviations of the mean fraction of defectors at time 
ti (depicted in (a) by the arrows) where the corresponding (adjusted) 
replicator dynamics predicts x{ti) = 0.5; (c) same for the variance 
of the mean fraction of defectors. The results for the simulations and 
the Langevin equation are in excellent agreement. Both the devia- 
tions and the variance converge to the deterministic dynamics with 
1/N (dotted line), as expected (b — 1, c = 0.5, w = 0.9, time scale 
for simulations 1/A'', averages over 10® realizations). 



(AH) = {H{t + 1) - H{t)), which is calculated for a sin- 
gle stochastic update step and averaged over many realiza- 
tions. In contrast, the adjusted replicator dynamics leads to 
H < 0, where equality holds only at the Nash equilibrium q. 
H represents a Lyapunov function with q as the unique and 
asymptotically stable interior fixed point. Interestingly, for fi- 
nite N, i.e. for the Moran process, the drift towards q changes 
sign: Below a critical population size Nc{w), q is unstable 
and the trajectories spiral towards the boundary. However, for 
N > Nc{'w), q becomes stable, cf. Fig.|2] Similar qualitative 
changes based on population sizes have recently been reported 
inRefs. 

In conclusion, we presented a mathematically consistent 
transition from descriptions of the microscopic processes rel- 
evant for individual based simulations over stochastic approx- 
imations of the dynamics in finite populations to a determinis- 
tic mean field theory governed by replicator dynamics. In par- 
ticular, we have shown that the intrinsic stochasticity arising 
in finite populations can be captured by a Langevin term in the 
replicator dynamics. This leads naturally to absorbing bound- 
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fully determined by the underlying microscopic processes. 
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FIG. 2: Battle of the sexes: (a) Average drift in a finite population 
for different population size A'^ and fixed w — 0.3 for the local up- 
date mechanism (A) and the Moran process (o). For (AH) < 0, 
the system spirals to the Nash equilibrium q; for (AH) > 0, it spi- 
rals towards the absorbing boundaries. For the local update process, 
AH — > with (see inset), which hints at the constant of mo- 
tion in the limit N oo (see text). For the Moran process, the 
system drifts towards the boundaries below a critical population size 
Nc (marked by the arrow) but approaches q for N > Nc- (b) The 
critical population size Nc (where (AH) ~ 0) decreases as a func- 
tion of the selection pressure w (N = Ng = Na, averages over 10^ 
realizations). 



aries in the resulting stochastic differential equation. Both, 
the microscopic processes and their stochastic approximation 
converge with 1 /N to the solution of the corresponding repli- 
cator equation. 

The frequency dependent Moran process 1 12] is intimately 
connected to the adjusted replicator dynamics |3]. Conversely, 
the proposed local update rule corresponds to a finite popula- 
tion equivalent of the standard replicator dynamics |4]. While 
the qualitative dynamics of the two approaches is the same, 
quantitative differences vanish only upon non-linear rescal- 
ing of time. However, for interacting (sub)populations, such 
rescaling is no longer possible and can give rise to entirely dif- 
ferent qualitative dynamics. In particular, for Dawkins Battle 
of the Sexes, we have shown that, for the Moran process, the 
stability of the mixed Nash equilibrium depends on the popu- 
lation size such that it becomes unstable below a critical size 
and no mixed population can persist. 

John Maynard Smith asked whether the adjusted or the 
standard replicator dynamics is more appropriate to describe 
evolutionary changes j3|. Here we have shown that this is 



* Electronic address: l| traulsen@fas.harvard.edu | 

[1] J. Maynard Smith and G. Price, Nature 246, 15 (1973). 

[2] P. Taylor and L. Jonker, Math. Biosci. 40, 145 (1978). 

[3] J. Maynard Smith, Evolution and the Theory of Games (Cam- 
bridge Univ. Press, Cambridge, 1982). 

[4] J. Hofbauer and K. Sigmund., Evolutionary Games and Popu- 
lation Dynamics (Cambridge Univ. Press, Cambridge, 1998). 

[5] M. A. Nowak and K. Sigmund, Science 303, 793 (2004). 

[6] Y. Sato and J. P Crutchfield, Phys. Rev. E 67, 0I5206(R) 
(2003). 

[7] S. Wright, Genetics 16, 97 (1931). 

[8] R. A. Fisher, The Genetical Theory of Natural Selection 
(Clarendon Press, Oxford, 1930). 

[9] P. A. P. Moran, The Statistical Processes of Evolutionary The- 
ory. (Clarendon, Oxford, 1962). 
[10] E. Lieberman, C. Hauert, and M. A. Nowak, Nature 455, 312 
(2005). 

[11] D. A. Kessler and H. Levine, Phys. Rev. Lett. 80, 2012 (1998). 
[12] M. A. Nowak, A. Sasaki, C. Taylor, and D. Fudenberg, Nature 
428, 646 (2004). 

[13] C. Taylor, D. Fudenberg, A. Sasaki, and M. A. Nowak, Bulletin 

of Math. Biology 66, 1621 (2004). 
[14] J. C. Claussen and A. Traulsen, Phys. Rev. E 71, 025101(R) 

(2005). 

[15] S. Karlin and H. M. A. Taylor, A First Course in Stochastic 

Processes. (Academic, London, 1975), 2nd ed. 
[16] For weak selection, w ^ 1, the fixation probability yields 



1 prol 

(1 + f {aN - 13)). In Ref. jll « and /? have 
been computed for the Moran process, a = a-\-2b — c — 2d 
and P — 2a + b + c — 4:d. For the local update mechanism, these 
parameters have to be multiplied by a factor 2 / ATTmax • 
[17] N. G. van Kampen, Stochastic Processes in Physics and Chem- 
istry. (Elsevier, Amsterdam, 1992). 
[18] H. G. Schuster, Complex Adaptive Systems (Scator, 

Saarbriicken, Germany, 2002). 
[19] D. Fudenberg and C. Harris, J. Econ. Theory 57, 420 (1992). 
[20] L. A. Imhof, Annals of Appl. Prob. 15, 1019 (2005). 
[21] D. Foster and P Young, Theo. Pop. Biol. 38, 219 (1990). 
[22] A. Traulsen, T. Rohl, and H. G. Schuster, Phys. Rev. Lett. 93, 
028701 (2004) . 

[23] J. Hofbauer and K. H. Schlag, J. Evol. Econ. 10, 523 (2000). 
[24] R. Axelrod and W. D. Hamilton, Science 211, 1390 (1981). 
[25] R. Dawkins, The Selfish Gene (Oxford Univ. Press, Oxford, 
1976). 

[26] In the economic literature fif, ^7^, the term "Battle of the 
Sexes" refers to a coordination game with two Nash equilibria 
rather than a cyclic game. 

[27] H. Gintis, Game Theory Evolving (Princeton Univ. Press, 
Princeton, 2000). 

[28] J. Weibull, Evolutionary Game Theory, (MIT Press, Cam- 
bridge, 1995). 

[29] L. A. Imhof, D. Fudenberg, and M. A. Nowak, Proc. Natl. Acad. 
Sci. 102, 10797 (2005). 



